Skip to content

Proof of concept: EDA control perturb - #856

Open
metdyn wants to merge 85 commits into
developfrom
feature/ygyu/superset_eda_Lanczos
Open

Proof of concept: EDA control perturb#856
metdyn wants to merge 85 commits into
developfrom
feature/ygyu/superset_eda_Lanczos

Conversation

@metdyn

@metdyn metdyn commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Description

After:

The ControlPert class from JEDI oops is used to execute EDA under the concept of incremental data simulation. As a proof of concept, this workflow framework is set up and named eda_controlpert_atmos.

Realistic runs with all observations, however, fail due to memory issues. I can only run a few conventional observations with the Control Pert method. Including IASI and other IR data will force NCCS nodes to reboot, so I have only included aircraft temperature and wind data in the eda_control_pert_tier_1_fast question default. We expect this issue be resolved in oops in the future.

To run EDA control pert, issue command:

swell create eda_controlpert_atmos
swell launch your_suite

The output data structure will contain files such as:


37K    Aug 4 17:35 jedi_eda_control_pert3D_config_chunk002.yaml
303K   Aug 4 17:35 jedi_eda_control_pert3D_log_chunk002.log
288M Aug 4 17:43 eda.mean-inc.20231010_000000z.nc4
36K   Aug 4 17:43  jedi_diffstates_output_latlon.log

eva/
└── increment
    ├── increment_eva.yaml
    └── map_plots
        ├── ps
        │   └── inc_ps.png
        ├── q
        │   ├── inc_q_1000.png

Note

I will continue to clean up the code, once #804 is merged.

Dependencies

Impact

Proof of concept only. Not for full observation runs.

metdyn and others added 30 commits May 27, 2026 13:43
…r_central_block(None))

+        self.jedi_rendering.add_key('saber_outer_block', self.config.saber_outer_block(None))
ens_mean_var read output from obs perturbed 3d-Var
    {% for i in range( 1, models[model_component]['ensemble_num_members'] + 1 ) %}
    {% for i in range(1, 5) %}
it is only 50% working
GetEnsembleGeosExperiment
qd.geos_x_ensemble_directory
qd.background_time_offset("PT3H")
ensemble_block.py
two_states.py
state_ensemble.py
IP.
  - select minimal state variables / field_io_names
  - add `if state == ensemble` selection
  - correct typo filenames -> filename
2. qd.window_length("PT6H"),
3. geos_atmosphere
   background_time_offset:
   default_value: PT9H
1. remove localensembleda from tier-1 test
2. disable localensembleda from code test
metdyn added 30 commits July 20, 2026 08:35
- use dict to replace string i by j for jedi config dict
nnode=3, nx/ny=4/5;  perhost=120
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant